|
|
|
|
|
|
|
|
Hello, Kinbay, Q1. Please let me know why we couldn't get the certification. Q2. You told me that "I fixed the problem with tvts_2.9_r1 to run the 3 new "Not Executed" and upload the report to Google." Could you please explain what you fixed? Thank you. Sangmin Choi. |
|
|
Hi Sangmin Q1: Some test cases did not execute in tvts_2.10r1. The APA server detected it, so the auto-approval system is blocked. Q2: As you say in the description the tvts_2.10r1 did not include and auto-execute some test cases. 3PL recommends that we use the tvts_2.9r1 to verify the execution of the test case. If executed upload the tvts_2.9r1 report to the APA server to fix it. The first time, show the one test case. After fixing the test case show the other 3 test case - TvsYouTubeTs.com.google.android.apsyts.tvts.YtsReportytsFunctionalTestsAssortedMediaRecrderAP+BFAB8332-5B9B-45C7.80A02A54B3EA3A5]@ameabiv7a - TvtsMediaLaunchTimesHostTestCases.com.google.android.apps.medialaunchtimes,.tvts.TvtsMediaaunchTimesHostTesttestPlayMoviesMediaStartTime@armeabiv7a - TvtsMlemoryScoreNewVideoTestCases.com.google.android.memory.tvts.TvtsMemoryScoreNewVideoTest.testMemoryScore@armeabi-v7a - TvtsPiaHostTestCases.com.pia.tvts.TvtsPiaHostTesttestStrongPlayintegrity@armeabi-v7a BR, Kinbay image.png (125,719 bytes) image-2.png (64,106 bytes) |
|
|
Hello, Kinbay, After fixing the test case show the other 3 test case -> What does "fixing test case" means? Does it means that running the TVTS using previous version(2.9_r1)? Or, did you modify something? Thank you. Sangmin Choi. |
|
|
Hi Sangmin What does "fixing test case" means? => Upload the report of testing the below test case with tvts_2.9r1. TvsYouTubeTs.com.google.android.apsyts.tvts.YtsReportytsFunctionalTestsAssortedMediaRecrderAP+BFAB8332-5B9B-45C7.80A02A54B3EA3A5]@ameabiv7a BR, Kinbay |
|
|
Hello, Kinbay, Jungyu Kim provided the report of testing the below test case with tvts_2.10r1. TvsYouTubeTs.com.google.android.apsyts.tvts.YtsReportytsFunctionalTestsAssortedMediaRecrderAP+BFAB8332-5B9B-45C7.80A02A54B3EA3A5]@ameabiv7a At that time, you told me that still can't not get approval. When you upload the report using tvts_2.9r1(TvsYouTubeTs.com.google.android.apsyts.tvts.YtsReportytsFunctionalTestsAssortedMediaRecrderAP+BFAB8332-5B9B-45C7.80A02A54B3EA3A5]@ameabiv7a), did you get the approval right away? Thank you. Sangmin Choi. |
|
|
Hi Sangmin When you upload the report using tvts_2.9r1(TvsYouTubeTs.com.google.android.apsyts.tvts.YtsReportytsFunctionalTestsAssortedMediaRecrderAP+BFAB8332-5B9B-45C7.80A02A54B3EA3A5]@ameabiv7a), did you get the approval right away? => Not get the approval but the "Not Execute" disappears immediately. The test with 2.10r1 did not work like 2.9r1 when uploading the report. I know it's illogical, but it works like this. BR, Kinbay |
|
|
Hello, Kinbay, Yes, I also think it's illogical. How about our next step? Could you ask it to 3PL or Google Issue tracker? Or, should we just wait to fix it by Google? Thank you. Sangmin Choi. |
|
|
Hi Sangmin I have already had a conversation with the 3PL. They know the problem, so please wait for them to fix it. BR, Kinbay |
|
|
Hello, Kinbay, Okay, I see. Let's check at next self MR. Best Regards, Sangmin Choi. |
|
|
Hi Kinbay, We have tested new TVTS tool (2.11_r1) for this issue. The result of test, com.google.android.apps.yts.tvts.YtsReport[FunctionalTestsAssortedMediaRecorderAPI] item is still not running. As I know that the 3PL is already know this issue. But the new TVTS is not fixed this issue. Could you please ask 3PL again? I have attached the TvtsYouTubeTS result file of TVTS 2.11_r1 Thank you. Kim |
|
|
|
|
|
Hi JunGyu a. Should we upload a test version to APA to scan 2.11r1 again and check that the issue still exists? b. If the issue still exists the 3PL advised us to follow the steps below to create an issue for Google. (1) Using the below command run with 2.9r1 and 2.11r1 tools to get two reports. ./tools/tvts-tradefed run commandAndExit tvts-cert -m TvtsYouTubeTS --module-arg TvtsYouTubeTS:set-option:target-tests:'Functional Tests Assorted MediaRecorder API' (2) Capture the bugreport. (3) Create an issue for Google to fix the APA detection. BR, Kinbay |
|
|
Hi Kinbay, a. Should we upload a test version to APA to scan 2.11r1 again and check that the issue still exists? => My opinion is, if there are no restrictions for upload test result to APA to check the issue, I think it would be a good idea to upload it and check the test results. => You can find FW for test to below path. => release_by_ALT/mantis-1996/k000c-542r3702_SU-20241022-BFX-AT100/ b. If the issue still exists the 3PL advised us to follow the steps below to create an issue for Google. => I have attached the result files with 2.9r1 and 2.11 r1. But on the test result, "Functional Tests Assorted MediaRecorder API" is marked as PASS. => Should we upload the Full test result that "Functional Tests Assorted MediaRecorder API" did not run? Thank you. Kim |
|
|
Hi JunGyu Can you supply other reports for 3702? I need to upload complete xts reports to check the issue. BR, Kinbay |
|
|
Hi Kinbay, Do you mean CTS / CTS on GSI / GTS / VTS / STS reports? If yes, it seems It will take some more time to supply the report. Thank you. Kim |
|
|
Hi JunGyu Yes, because the APA scanner will not process if only the tvts. BR, Kinbay |
|
|
Hi Kinbay, I have uploaded xTS results and BTS FW. FW version : 15.543.21 BTS FW : http://altserver01.duckdns.org/release_by_ALT/BPM-38104/SKB~BFX_AT100~BFX-AT100~12~STTC.220724.001~15.543.21-20241118~user~release-keys.zip FYI, There are 2 FAILs in this result. GTS : GtsMediaRouterTestCases / com.google.android.gts.media.router.MediaRouter2Test#publishAddresslessBluetoothRoute_doesNotCrashSystemUi TVTS : TvtsSmokeTestCases / com.google.android.smoketests.tvts.TvtsSmokeTest#testSmoke[smoke-json-file-exists] I don't think this will cause any problems in verifying this ticket. Thank you. Kim |
|
|
Hi JunGyu Your report package is missing CTS_VERIFIER. Please provide it. BR, Kinbay |
|
|
Hi JunGyu I think the tvts smoke test also needs to be provided to check the ticket. Because the APA will detect that the smoke test is missing, it will not show the other failures on TVTS. Until the smoke test is fixed, the other TVTs will not appear. BR, Kinbay |
|
|
Hi Kinbay, I have attached the result files of TVTS, CTS-Verifier. The case of TVTS, I have proceed smoke test. So, there are only 1 failer of GTS - waiver contents. Please check it. Thank you. Kim |
|
|
|
|
|
Hi Kinbay, I have re-attached TVTS result because the result previous I attached is included "userdebug" Please use this file and I'm sorry about inconvenience. Thank you. Kim |
|
|
Hi JunGyu I believe the ticket issue has been resolved regarding the "Not Executed" status not being detected. Not get the "Approval" because of the GTS waive problem that needs to wait for the manual approval. Does the version report need to continue to get approval or this is just to test the issue? BR, Kinbay image-3.png (60,114 bytes) image-4.png (64,252 bytes) |
|
|
Hi Kinbay, This FW's purpose is to check the com.google.android.apps.yts.tvts.YtsReport[FunctionalTestsAssortedMediaRecorderAPI] item. So this version is not needed to get approval. Is there any posibility that it is marked as Not executed due to GTS fail item? If not related, I think we can close this issue. Thank you. Kim |
|
|
Hi Kinbay, Is there any update? Thank you. Kim |
|
|
Hi JunGyu Please provide the smoke test sheet link. BR, Kinbay |
|
|
Hi Kinbay, Smoke test link is below. https://docs.google.com/spreadsheets/d/1iS5AKCSIPEc7Bi6zmgN1gwMEJsU3YALazLcAuKoD9dE/edit?pli=1&gid=900943338#gid=900943338 Thank you. Kim |
|
|
Hi JunGyu Is there any posibility that it is marked as Not executed due to GTS fail item? => The TVTS "Not Executed" is not related to GTS failure. So I think the problem is solved. BR, Kinbay |
|
|
Hi Kinbay., Thank you for confirmation. I close this issue and I will let you know if any problem. Thank you Kim |
- yk.jung ( (ALTech) Younkwang Jung ) developer
.png)
.png)
.png)
